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® Method and apparatus for automated assay of biological specimens. 



@ An apparatus and method for automated assay 
of biological specimens positioned on microscope 
slides. The apparatus comprises an interactive op- 
tical subsystem (11a) for viewing the biological 
specimen on the slide and for producing an Inter- 
active video signal corresponding to the viewed im- 
age. An automated optical subsystem (1 1 b) Includes 
a single high power microscope objective for scan- 
ning a rack of slides, portions of which having been 
previously identified for assay in the interactive op- 



tical means (11 a). The system also includes a pro- 
cessor for processing the interactive and automatic 
video signals for the two optical subsystems. The 
processor receives the automatic video signal and 
performs biological assay functions upon it. A meth- 
od and apparatus are also disclosed for marking 
points for later analysis on the microscope slides 
and for associating an analysis function with each 
marked point. 
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CROSS REFERENCE TO RELATED APPLICA- 
TION 



This is a continuation-in-part of application 
Serial Number 764,336 filed September 23. 1991 to 
Bacus, et al. 

BACKGROUND OF THE INVENTION 

This invention relates to a system for perform- 
ing an assay of slide mounted biological cell sam- 
ples, and more particularly, for providing an auto- 
mated method and arrangement to identify to the 
system locations at which to perform selected ana- 
lysis functions. 

The diagnosis and/or prognosis of a patient's 
condition frequently includes the removal of a cell 
sample, such as a tissue mass, from the patient. 
Although an attending physician may have good 
intuition regarding the patient's diagnosis and/or 
prognosis, confirmation of the diagnosis with histo- 
logical examination of the cell sample removed 
from the patient is necessary. The histological ex- 
amination entails cell staining procedures which 
allow the morphological features of the cells to be 
seen relatively easily in a light microscope. A pa- 
thologist, after examining the stained cell sample, 
makes a qualitative determination of the state of 
the tissue and reaches a conclusion regarding the 
prognosis for the patient. While this diagnostic 
method has a long history, it is somewhat lacking 
in scientific rigor since it is heavily reliant on the 
subjective judgment of the pathologist and it is 
extremely time-consuming. 

The alternative to the strictly qualitative and 
time-consuming human analysis is automated cell 
analysis where the pathologist uses specialized 
equipment to perform the analysis. Flow cytometry 
equipment is one type of automated apparatus for 
cell analysis. With flow cytometry, mass tests are 
performed in gross on a specimen cell population 
without a researcher being able to exclude or in- 
clude certain data of the population. The specimen 
is measured '*as is** without really knowing what 
cells are being measured and how many. Important 
single cell data or data from relatively small groups 
of cells are lost in the overall averaging of a speci- 
men. Further, relatively large amounts of a speci- 
men Irave to be used to provide a required level of 
accuracy. Again, small changes in individual cells 
or small cell populations cannot be discerned. 

Commercially available general purpose flow 
cytometers are very expensive and can handle 
only liquid blood specimens or tissue disaggregat- 
ed specimens. Additionally, flow cytometers are 
incapable of working on standard tissue sections or 
using conventional microscope slides which are the 
preferred specimen forms of pathology lat)orato- 



ries. 

Although the automation of cell analysis using 
microscope slide cell samples is exceedingly dif- 
ficult, such has been automated to a human-ma- 

5 chine interactive level. One such method and ap- 
paratus is described in United States Patent No. 
4,471 ,043 to Bacus. for Method And Apparatus For 
Image Analysis of Biological Specimens. Cell sam- 
ples are attached to slides and an operator adjusts 

JO the system optics to view desired image fields of 
the cell sample. The operator then selects and 
classifies particular cell objects of the sample. After 
such operator action, the automated equipment 
quantitatively measures particular attributes of the 

'5 selected and classified cell objects and records a 
digital representation of the optical image. The 
measured attributes can be reported on a per ob- 
ject basis or on an accumulated basis, and the 
stored image representations can later be read 

20 form memory for review. 

Heretofore, slides prepared for use in image 
analysis often have different areas widely spaced 
on the" slide at which different actions or functions 
are to be performed. For instance, a slide may 

25 have a clear area thereon which is selected for use 
in adjusting or calibrating the light and color levels 
needed for analysis of specimens located on the 
slide in another area of the slide. The operator may 
want to adjust the light level for each slide or for a 

30 group of slides in a slide tray or carrier. Also, the 
specimen cells may be located in a small portion 
of a specimen area on the slide and considerable 
time would be saved if the carrier positioned the 
slide specimen with this small portion of specimens 

36 directly beneath the microscope rather than having 
to examine large empty areas before locating the 
small specimen. Likewise, a calibration specimen in 
a calibration area may take a long time to locate by 
the machine because of the large empty area 

40 about the specimen. Still in other slides such as 
blood slides, the blood sample may cover the 
entire slide leaving no clear area for light setting. 

The slides can be searched by the analysis 
apparatus to find particular areas, such as speci- 

45 men or calibrate areas, however, doing so with a 
for example. 40x objective used for analysis, is 
difficult and time-consuming. The image area of a 
high power objective is only on the order of 20 by 
20 microns so that each step in a search routine is 

50 very short and many such steps must t>e per- 
formed. Further, the depth of field with a high 
power objective is very short and refocusing must 
occur at each image field. When a clear area is 
under the slide, as will be the case for most of a 

55 search for small samples, much time is wasted 
trying to focus on non-existent objects. It is a 
significant problem in an automated analysis to be 
able to locate a small area or a specimen on a 
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slide and to perform a desired function on each of 
a plurality of areas on the slides in a carrier in a 
fast and efficient manner. 

SUMMARY OF THE INVENTION 

In accordance with the present Invention, an 
operator first performs a visual inspection of each 
slide noting the location of each area on the slide 
that an operator wants to perform a function. The 
operator then stores the address of the noted loca- 
tion and the associated function in a data file of the 
apparatus. The operator is assisted in identifying 
the areas on the slide by a coordinate system 
which may be in the form of a coordinate mask or 
pattern shown on or through the slide. By moving a 
computer monitor cursor to a point on a slide 
image, the address location of the point may be 
entered into the computer data file. Preferably, the 
operator may more finely define the coordinate 
address of the point by having an enlarged grid 
area shown on the monitor. The operator may then 
locate a spot on the enlarged grid to enter its 
coordinate location in the data file. The operator 
then enters the selected function for that location 
into the data file in association with the location. 
The operator proceeds to one or more locations on 
the same slide and enters a location and an asso- 
ciated function therefore for that slide. The operator 
proceeds with such a review for each slide of the 
group of slides in the carrier. After having consid- 
ered each slide and associated a selected function 
for each selected location, the data file is com- 
pleted. The carrier is then positioned in automated 
analysis apparatus for automatic analysis. The ap- 
paratus reads the data file and moves the carrier to 
have each slide and each location on each slide 
positioned beneath the microscope. The image 
analysis means performs the designated function at 
each location on each slide, and then slide-by-slide 
until all of the slides in the carrier have been 
automatically analyzed. 

The invention includes an operator interactive 
method of and apparatus for easily finding very 
small things, such as a small dot or sliver of 
specimen, on a very large slide area that would be 
very difficult and time-consuming to locate auto- 
matically at the high resolution, e.g.. 40x used 
when using an image analysis microscope. The 
slides used are transparent and the operator can 
easily see a zero, zero location point and a small 
sliver or dot of specimen, e.g.. one centimeter in 
length or diameter in a targe specimen area on the 
slide, e.g.. a 5 x 20 centimeter specimen area. The 
field of the 40x microscope may be in the order of 
a 20 X 20 micron field and a large number of 
focusing repetitions and stepping of the stage and 
slide would be necessary to cover the empty areas 



if the specimen location were not located and ad- 
dressed under low resolution by an operator pre- 
vious to the slide being scanned automatically by 
the image analysis machine. The operator visually 

5 sees the specimen in a particular location on the 
slide and designates the section of the slide having 
the specimen and notes a marked position on that 
designated section as the coordinate address for 
the specimen. Preferably, the operator views a grid 

10 projected on the siide and designates the grid 
section having the specimen and such designation 
causes the grid section and specimen to be shown 
in enlarged size, e.g., 4x or lOx on a monitor 
screen. The mark noted by the operator may have 

15 been placed there by the pathologist at earlier time 
or the mark may be made by operator moving a 
mark along the monitor screen to a position adja- 
cent the specimen while it is being viewed. In 
either event, the operator enters the address of the 

20 mark as the location to begin the scanning opera- 
tion for the specimen. The preferred apparatus 
includes a flat bed carrier having a plurality of 
slides therein and the operator prepares each slide 
usually with several addresses and a specific func- 

25 Won for the address such as focus, scan, analyze, 
etc. There may be more than one operator and 
more than one station that is pre-screening the 
slides interactively and the slides in their respective 
carriers are transferred to the automatic image ana- 

30 lysis machine which automatically steps the flat 
bed carrier or a carrousel-type carrier to quickly 
move each slide through the various functions 
needed to perform the analysis. 

35 BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a perspective view of an apparatus for 
assaying biological specimens embodying the 
present invention; 

40 FIG. 1A is a perspective view, having portions 
broken away, of an automatic optical input sub- 
system of the apparatus for assaying biological 
specimens shown in FIG. 1; 
FIG. 2 is a block diagram of the apparatus 

45 shown in FIG. 1; 

FIG. 3 shows a slide holder and associated 
control equipment of the apparatus shown in 
FIG. 1: 

FIG. 4 is a block diagram view of focus and light 
50 control portions of the apparatus shown in FIG. 
1; 

FIG. 5 is a plan view of a tissue section micro- 
scope slide for use with the apparatus of FIG. 1; 
FIG. 6 is a graphical representation of the op- 
55 tical properties employed in biological specimen 
assay; 

FIGS. 7 and 8 are schematic views of the bio- 
logical specimen preparation prior to assay; 
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FIGS. 9 and 10 are flow diagrams of the control 
procedures invoked In tfie assay of a plurality of 
biological specinnens; 

FIG. 1 1 represents an image field of an optically 
unfiltered tissue section; 

FIG. 12 represents the image field of FIG. 11 
when optically filtered by a red filter having a 
passband centered about 620 nanometers: 
FIG. 13 represents the image field of FIG. 11 
when optically filtered by a green filter having a 
passband centered about 500 nanometers; 
FIG. 14a is a template for use in identifying slide 
action points; 

FIG. 14b is a representation of a slide tray in 
which the template of FIG. 14d is visible; 
FIG. 15 shows the apparatus operations avail- 
able from a tray setup function of the apparatus; 
FIG. 16 is a visual image presented to an oper- 
ator during the tray setup functions; 
FIG. 17 shows a data file established during tray 
setup: 

FIG. 18 is a flow diagram of a setup scan 
routine; 

FIG. 19 shows a monitor screen displayed during 
tray setup; 

FIG. 20 shows a monitor screen displayed dur- 
ing a setup scan routine: 

FIG. 21 is a flow diagram of analysis by the 
apparatus after slide locations have been se- 
lected; 

FIG. 22 is a flow diagram of operations per- 
formed to resolve joystick actions; and 
FIG. 23 is a flow diagram of a scan routine. 

DETAILED DESCRtPTiON OF THE PREFERRED 
EMBODIMENT 

The preferred embodiment disclosed herein is 
used for the assay or quantitation of biological 
specimens specifically estrogen and progesterone 
in tissue samples. The tissue sample staining and 
measuring techniques for the 

estrogen/progesterone assay are described in de- 
tail in United States Patent No. 5.008.185. issued 
April 16, 1991 to Bacus which is hereby incor- 
porated by reference. The tissue sample assay is 
performed using a two-color optical system to en- 
hance the optical characteristics of stained tissue 
samples. It will be apparent to those skilled in the 
art that many Inventive features of the disclosed 
embodiment may be employed for other types of 
cell analysis e.g., DNA quantification and that other 
types of optical apparatus e.g.. single color could 
be employed. 

An apparatus for assaying biological speci- 
mens, and embodying the present invention and 
generally identified by numeral 10 is shown in 
perspective view in FIG. 1 and in block diagram 



form in FIG. 2. The apparatus 10 comprises an 
interactive optical input system 11a primarily for 
use in low power scanning of microscope slides of 
biological specimens to select fields for later analy- 

5 sis. An automated assay processing system lib 
also comprises a portion of the apparatus for scan- 
ning up to eight slides at once at relatively high 
magnification for performing biological assays on 
the slide. A processor system 32 receives signals 

10 from the optical units for later image processing. 

The interactive optical system 1la comprises 
an optical microscope 12, which may be of any 
conventional type, but in this embodiment, is a 
Riechart Diastar. An optical conversion module 14 

;5 is mounted on the microscope 12 to enhance the 
optically magnified image of cell samples viewed 
with the microscope 12. The optical conversion 
module 14, as may best be seen in FIG. 2. in- 
cludes a beam-spHlting prism 80 which conveys 

20 approximately 90% of the light into optical conver- 
sion module 14 and passes the remaining 10% to 
a microscope eyepiece 76. The light transmitted 
into module 1 4 is fed to a dichroic beam-splitter 82 
which reflects a portion of the light to a television 

25 camera 20 via a red filter 18 and a mirror 81. The 
remaining portion of the light is filtered by a dich- 
roic beam-splitter 82 and fed to a television camera 
26 through a green filter 24. The dichroic beam- 
splitter 82 selectively passes light having 

30 wavelengths greater than approximately 560 nano- 
meters to the filter 18 and having a wavelength of 
less than 560 nanometers to the filter 24. Thus, the 
dichroic t)eam-splitter 82 acts as a first color filter 
before the light reaches the color filters 18 and 24. 

35 Red filter 18 is a 620 t 20 nanometer bandpass 
optical transmission filter which provides a high 
contrast image to the camera 20. As shown in FIG. 
2. the camera 20 then generates an NTSC image 
signal which is fed through an optical signal switch 

40 90a to an image processor 90 of an image proces- 
sor module 28 (FIG. 2). Green filter 24 is a 500 ± 
20 nanometer narrow bandpass optical transmis- 
sion filter which provides a high contrast image to a 
camera 26. The camera 26 then feeds an NTSC 

45 image signal through the optical signal switch 90a 
to an image processor 92. Both of the image 
processors 90 and 92 contain analog to digital 
converters for converting the analog NTSC signals 
to a digitized 384 by 485 array pixel image. The 

50 center 256 by 256 array of pixels from this digitiz- 
ed image is then stored within frame buffers inter- 
nal to the image processors 90 and 92. The visual 
image represented by the 256 by 256 array of 
pixels is referred to as an image field. 

55 During assembly of the apparatus of FIG. 1. 

and from time to time thereafter, if necessary, the 
optical elements of conversion module 14 are ad- 
justed so that each camera 20 and 26 receives the 
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same optical image and each pixel of the digitized 
pixel arrays produced by processors 90 and 92. 
presents the same region of a viewed optical field. 

Each of the image processors 90 and 92 is a 
Model AT428 from the Data Cube Corporation, and 
includes six internal frame buffers. The image pro- 
cessors 90 and 92 are connected to a system bus 
34 of a computer 32. The frame buffers of image 
processors 90 and 92 are mapped into the address 
space of a microprocessor 36 in computer 32 to 
provide easy access for image processing. Addi- 
tionally, an image monitor 30 is connected to im- 
age processor 92 and displays a cell sample image 
field stored in a predetermined one of the frame 
buffers. The storage of an image field representa- 
tion into the predetermined frame buffer is de- 
scribed later herein. 

The automatic optical conversion module lib, 
as may best be seen in FIG. 2. includes, a prism 
80a which conveys the light into optical conversion 
module I4a. The light transmitted into module I4a 
is fed to a dichroic beam-splitter 82a which reflects 
a portion of the light to a television camera 20a via 
a red filter 18a and a mirror 81a. The remaining 
portion of the light is filtered by a dichroic beam- 
splitter 82a and fed to a television camera 26a 
through a green filter 24a. The dichroic beam- 
splitter 82a selectively passes light having 
wavelengths greater than approximately 560 nano- 
meters to the filter 18a and having a wavelength of 
less than 560 nanometers to the filter 24a. Thus, 
the dichroic beam-splitter 82a acts as a first color 
filter before the light reaches the color filters 18a 
and 24a. Red filter 18a is a 620 ± 20 nanometer 
bandpass optical transmission filter which provides 
a high contrast image to the camera 20a- As shown 
in FIG. 2, the camera 20a then generates an NTSC 
image signal which is fed through the optical signal 
switch 90 to the image processor 90 of the image 
processor module 28 (FIG. 2). Green filter 24a is a 
500 t 20 nanometer narrow bandpass optical trans- 
mission filter which provides a high contrast image 
to a camera 26a. The camera 26a then feeds an 
NTSC image signal through the optical signal 
switch 90a to the image processor 92. 

The microprocessor 36 of computer 32 is an 
Intel 80486 microprocessor which is connected to 
the system bus 34. The optical switch 90a. under 
control of the microprocessor 36. selects the signal 
from interactive unit 1la or automatic unit lib to 
be fed to the image processors 90 and 92. A 
random access memory 38 and a read only mem- 
ory 40 are also connected to the system bus 34 for 
storage of program and data. A disk controller 42 is 
connected by a local bus 44 to a Winchester disk 
drive 46 and to a floppy disk drive 48 for secon- 
dary information storage. Advantageously, local bus 
44 is connected to a moveable media bulk data 



drive 45 such as an optical write once read many 
times (WORM) drive for image field recording and 
retrieval. 

A video conversion board 50, in this emtxjdi- 

5 ment a VGA board, is connected to the system bus 
34 to control an instruction monitor 52 connected 
to the VGA board 50. Operational information such 
as selection menus and reports of analysis are 
displayed on instruction monitor 52. A keyboard 

10 processor 54 is connected to the system bus 34 to 
interpret signals from a keyboard 56 connected to 
the keyboard processor 54. Input signals to micro- 
processor 36 are also generated by a hand control 
drive (mouse) 13 having a control button 15. Sig- 

/5 nals from mouse 13 and its button 15 are con- 
veyed to bus 34 via a mouse interface 17. A printer 
58 is connected to the system bus 34 for commu- 
nication with microprocessor 36. The apparatus 10 
also includes a joystick control device 13a of a 

20 type well known in the art. Signals from the joystick 
13a are conveyed to bus 34 via a joystick interface. 
17a, 

The automated image input subsystem lib of 
apparatus 10 performs automated X-Y slide posi- 

25 lioning. image focusing, light intensity adjustment 
and light color balancing functions. The X-Y slide 
position controlling apparatus is shown in FIGS. 1, 
1A, 3 and 4, and includes a slide holder 62a 
capable of holding eight microscope slides 101 

30 through 108 in side-by-side relationship such that 
the upper surfaces of the slides are substantially 
coplanar. Slide holder 62a. which is sometimes 
referred to as a flat bed carrier, is movably at- 
tached to the stage 65a of microscope objective 

35 64a by means of a slide holder base 63a. The 
portion of slide holder 62a positionable with respect 
to microscope objective 64a is controlled by an X 
position stepper motor 110 and a Y position step- 
per motor 111 which are mechanically attached to 

40 base 63a. The stepper motors 110 and 111 are of 
the type known in the art which respond to pulse 
signals from a slide holder position controller 60. 
The actual X and Y positions of the slide holder 
62a are sensed by an X position sensor 68 and a Y 

45 position sensor 66, respectively, which substantially 
continuously report position information to slide 
holder controller 60. In the present emtx)diment the 
slide holder 62a, base 63a, and position sensors 66 
and 68 including limit switches and numbered 110 

50 and 111 comprise a commercially available unit 
from Marzhauser Wetzlar GmbH Model EK8B-S4 
with Model MCL-3 control units. 

Responsive to appropriate stepper motor con- 
trol signals, the slide holder base 63a is capable of 

55 placing substantially all of each of slides 101 
through 108 under the objective 64a. Slide holder 
position controller 60 is connected to system bus 
34 by means of a communication path 61. Micro- 
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processor 36. as discussed later herein, transmits 
commands to slide holder position controller 60 
specifying an X and Y position to place under the 
microscope objective 64a. Slide holder position 
controller 60 responds to such commands by 
transmitting to the X and Y stepper motors 1 10 and 
lit the appropriate sets of putse signals to move 
the slide holder 62a to the desired X-Y position. 
The actual position of slide holder 62a is checked 
by slide holder position controller 60 during and at 
the completion of movement. The slide holder posi- 
tion controller 60 also maintains an internal record 
of the X and Y position of the slide holder 62a 
which internal record can be read by microproces- 
sor 36 via bus 34 and communication path 61 . 

The apparatus 10 also includes a focus and 
light controller 73 which controls the light intensity 
and color balance from the light source 84a. as 
well as the focus of the image field presented to 
microscope 12. (Vlicroprocessor 36 communicates 
with focus and light controller 73, via the system 
bus 34 and a communication path 74, to control the 
focus and light properties. FIG. 4 is a functional 
block diagram of focus and light controller 73 and 
its connection to objective 64a and to bus 34. The 
objective 64a includes a focus stepper motor 75a, 
which is controlled by focus and light controller 73 
through the stepper motor controller 73a to raise 
and lower the stage 62a. and thereby raise and 
lower the microscope slides 101 through 108 car- 
ried by slide holder 62a. IWicroprocessor 36 in- 
cludes a focus routine which is periodically per- 
formed during tissue analysis. When the focus rou- 
tine is entered, microprocessor 36 reviews a digital 
representation of an mage field from the image 
processors 90 and 92, and issues a command to 
focus and light controller 73. to raise or lower the 
stage by a specified amount. Focus and light con- 
troller 73 responsively transmits to focus stepper 
motor 75a electrical signals to implement the re- 
quested stage movement. By continued checking 
of the quality of the image field and adjustment of 
the up and down position of the slide holder 62a, 
microprocessor 36 brings the upper surface of the 
slide under the objective 64a into focus. 

Microprocessor 36 also stores a target value 
for the light intensity which is to be maintained 
during tissue sample analysis. This stored light 
intensity value is used by microprocessor 36 in 
conjunction with an intensity function to regulate 
the intensity of light from light source 84a. When 
the intensity function of microprocessor 36 is en- 
abled, the light intensity as represented by image 
fields from image processors 90 and 92 is deter- 
mined. Any departure from the stored target light 
intensity value is corrected by sending intensity 
control commands to focus and light controller 73 
which responds thereto by controlling a voltage 



regulator to increase or decrease the voltage ap- 
plied to light source d4d. Voltage regulator 83 may 
be. for example, a standard rotatable voltage regu- 
lator which is rotated by a stepper motor operating 

5 under the control of electrical signals from focus 
and light controller 73. 

The analysis performed in the present embodi- 
■ ment relies on a two-color system. For accuracy of 
measurement, it is important that the two-colors 

w observed by cameras 20 and 26 be of substantially 
the same intensity. The microprocessor 36 in- 
cludes a color balance function, which is called to 
match the intensities of the red and green colors 
applied to cameras 20 and 26 by light source 84. 

/5 Objective 64a has associated with it a condenser 
85a. which is controlled by a stepper motor 86a. 
electrically connected to focus and light controller 
73. When in the color balance function, micropro- 
cessor senses color Imbalance by comparing the 

20 image field of image processor 90 with that of 
image processor 92. Microprocessor 36 sends coj- 
or balance adjustment commands to focus and 
light controller 73 in response to color imbalance 
and through a process of repeated comparison and 

25 color balance adjustment cornmands achieves a 
color balance suitable for tissue sample analysis. 

FIG. 5 shows a representation slide e.g.. 102 
which has been prepared for analysis in accor- 
dance with the embodiment. The slide 102 includes 

30 a dark line outline 87 of a rectangle near one end 
thereof. The rectangle 87, which is printed in sub- 
stantially the same position on all slides is used 
during focus and tight adjustment routines and dur- 
ing slide preparation to identify the placement of 

35 tissue samples. 

A method of quantitating nuclear proteins of 
the present embodiment includes proyiding speci- 
men 88 cell objects on slide 102 and staining them 
with an optical enhancement factor which specifi- 

40 cally binds to the nuclear protein. The stain is then 
viewed with the image analysis system 10 to mea- 
sure the optical density of the stain for intensity 
measurement and to locate the areas in which stain 
is found for distribution measurement. Because the 

45 intensity of the staining relates to the quantity of 
the nuclear proteins, measurement of the different 
optical densities of the stain permits a direct mea- 
surement of the quantity of the proteins. In the 
preferred embodiment, control cell objects 89 are 

50 placed on a reference section of the slide 102 to 
provide a normalization or reference optical density 
for the staining. Further, one or several counter- 
stains can be used to further distinguish among 
several features of the celt objects. 

55 No actual marks except rectangle 87 appear on 

the slides 101-108 to differentiate the control cell 
objects 89 from the specimen 88. However, for 
automated location of points to begin analysis, the 
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Slide should be prepared such that the control cell 
objects 89 overlap the longitudinal center line of 
the slide (denoted by a dotted tine 91 in FIG. 5) 
and that the control cell objects 89 are closer to the 
rectangle 87 than is the specinnen 88. Similarly, the 
specinnen 88 should be positioned overlapping the 
longitudinal center line and farther from rectangle 
87 than the control cell objects 89. 

Preferably, in one particular embodiment, the 
staining method employs a sensitive peroxidase- 
anti peroxidase technique for visualization of es- 
trogen or progesterone receptors in specimens 
through the use of monoclonal antibodies directed 
specifically against those receptors. A diagram- 
malic representation of the process on the micro- 
scopic level is illustrated in FIGS. 7 and 8. Two 
portions of a human tumpr specimen containing a 
cell population from which the estrogen receptors 
are to be measured are placed on the two separate 
sections of the slide 14 and suitably fixed thereto 
as by tissue adhesive. The separate sections are 
then fixed in separate washes of formalin, methanol 
and acetone, and thereafter, treated with a blocking 
reagent to prevent non-specific binding of the sub- 
sequent reagents. 

The part of the specimen cells 88 to be mea- 
sured is incubated with a primary antibody, a mon- 
oclonal antibody (rat) to human estrogen receptor 
in the specimen portion of the slide. This antibody, 
as represented at 128, binds specifically to the 
estrogen receptor sites ER of this tissue portion. 
The reference portion 89 of the specimen is in- 
cubated with a control, normal rat IgG. represented 
at 130. The purpose of the control 130 is to evalu- 
ate the amount of binding of the immunoperoxidase 
reagents in this technique to nonspecific sites NS 
of the specimen to yield a background measure- 
ment. 

Both tissue sections 88 and 89 on the slide 
102 are then incubated with a bridging antibody, an 
anti-rat immunoglobulin (goat) illustrated at 132 in 
both figures. The bridging antibody 132 binds to 
the rat antibody 128 against' human estrogen re- 
ceptor in the specimen section 88 and to any 
bound normal rat IgG 1 30 in the control section 89. 

A rat PAP complex 134 is added to both sec- 
tions 88 and 89 of the specimen and binds to the 
anti-rat IgG bridging antibody at 132. After this 
step, a solution containing hydrogen peroxide and 
diaminobenzidine (DAB) and 4 N HCI is added to 
the specimen and control sections. The reaction of 
the peroxidase with hydrogen peroxide converts 
the bound DAB present into an insoluble reddish 
brown precipitate. The proportion of the precipitate 
and its location are influenced by the binding posi- 
tions of the PAP complex and, through the bridging 
and primary antibodies, the locations and amounts 
of the estrogen receptors in the specimen. 



The concentrations, timing, and chemical com- 
positions of the reagents used in this staining 
method are more fully described in the aforemen- 
tioned United Stales Patent No. 5.008,185. Prefer- 

5 ably, the monoclonal antibody which is used to 
bind to the estrogen receptor sites is one of those 
developed at the University of Chicago and des- 
ignated H222 sP2 and H226 sP2. and that which is 
used to bind to the progesterone receptor is one 

10 which is commercially available from Transbio Sari 
6 Rue Thiers. Paris France and designated mPRI. 

The DAB precipitate is then visualized by im- 
age analysis with apparatus 10 to determine the 
quantitation of the estrogen receptors in the speci- 

15 men. In general, the brown precipitate does not 
transmit light well and will show up as dark areas in 
the cells of the specimen. The optical density and 
hence pixel intensity will be related directly to the 
amount of DAB precipitate and to the quantity of 

20 estrogen receptor which has bound the antibodies. 
To be able to more clearly visualize the nuclear 
area of each cell, a counterstain of methyl green is 
added. It is important to note that both the primary 
slain of DAB precipitate and the counterstain of 

25 methyl green are specific to the nucleus of each 
cell. This means that debris and other cellular 
features will appear lighter in the microscope im- 
age and can be distinguished. 

A dual camera method is thereafter applied to 

30 distinguish Ihe areas stained by the DAB and the 
areas stained by the methyl green. The red and 
green filters 18 and 24 respectively form monoch- 
romatic images of the cell objects at their respec- 
tive cameras 20 and 26 which images can be 

35 stored in the apparatus 10. These images, one by 
the red filter and the other by the green filter, are 
used to separate the primary stained areas from 
the nuclear areas, and to separate the nuclear 
areas from other cell or field features. 

40 The results and desirability of this dual filtering 
of a counterstained cell image are more fully illus- 
trated in FIG. 6. The percentage of light transmitted 
through the nuclei stained with methyl green Is 
shown in the curve A as a function of the 

45 wavelength of light. The percentage of transmission 
of light for diaminobenzidine (DAB) is shown in 
curve B as a function of the wavelength of light. 
The bandwidth of wavelengths of light passed by 
the green filter is illustrated in band C while the 

50 bandwidth of wavelengths of light passed by the 
red filter is illustrated in band D. 

When an image of a counterstained cell popu- 
lation or specimen is filtered with the green filter 
24, substantially all of the areas stained with the 

55 methyl green will be invisible. This is because the 
methyl green curve A has a relative transmissive 
peak near this wavelength band while the dia- 
minobenzidine curve B is relatively non-transmis- 
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sive in this band. Thus, the areas with primary DAB 
Stain can be separated from the nuclear areas. At 
the other extreme of the graph, the band D of the 
red filter is positioned at a place where just the 
opposite occurs. The methyl green curve A has a 
relatively non-transmisslve valley in this bandwidth 
while the diaminobenzidine curve B is also rela- 
tively non-lransmissive. Thus, the nuclear areas 
containing both the primary stain and the counter- 
stain appear darker than other cell features and can 
be readily identified. 

Because of the relative differences in light 
transmission between the primary and counterstain 
in the two filtered bandwidths. the methyl green 
stained area is enhanced during one filtering rela- 
tive to other areas of the cell, and the areas which 
have diaminobenzidine precipitate are enhanced 
relative to the methyl green areas during the other 
filtering. Thus, the nuclear areas of the cell objects 
are optically enhanced along with the areas having 
DAB precipitate. 

FIG. 11 represents an optical image field of a 
tissue sample as presented to the microscope ob- 
jective 64. In the image of FIG. 11. objects 200. 
202 and 204 are green cell nuclei where nuclei 202 
and 204 have brown areas 206 and 208 respec- 
tively of DAB precipitate. Nucleus 200 does not 
contain any estrogen receptor and therefore, does 
not have any DAB precipitate. Objects 210. 212 
and 214 are various other cell features or debris 
from the tissue section. FIG. 12 represents the 
image field presented by red filter 18 to camera 20 
and its associated image processor 90. In FIG. 12, 
the nuclei 200, 202 and 204 stand out because of 
the counter staining and filtering, while the DAB 
areas are not visible. FIG. 13 represents the image 
field presented by green filter 24. to camera 26 and 
its associated image processor 92. In FIG. 13. the 
estrogen receptor areas 206 and 208 stand out so 
that their area and density can be readily mea- 
sured. The dotted lines of FIG. 13 represent the 
boundaries of nuclei 200, 202 and 204, which are 
shown for reference purposes. As previously dis- 
cussed, a monitor 30 is available for displaying 
image fields of the cell sample. When an image 
field is to be displayed, microprocessor 36 com- 
putes a composite image by summing the image 
field of image processor 90 (of the type shown in 
FIG. 12). With the image field of image processor 
92 (of the type shown in FIG. 13), the composite 
image is then stored by microprocessor 36 in the 
predetermined frame buffer of image processor 92 
which is the source of images for monitor 30. 

While the implementation shows a convenient 
and advantageous method for discriminating be- 
tween the two areas having counterstaining, it is 
recognized that there are various other staining or 
optical enhancement methods and filtering meth- 



ods which can be used to optically enhance one 
particular area or feature over another cell feature. 
For the quantitation of the specific hormonal recep- 
tors shown (progesterone and estrogen receptors). 

5 what is important is to distinguish the nuclear area 
which contains receptors by the presence of the 
diaminobenzidine precipitate. 

In the preferred embodiment, up to eight slides 
mounted in a slide tray are analyzed in an auto- 

10 matic analysis session. One of the eight slides 
(101) is a calibrate slide and the remaining seven 
slides are preferably prepared using tissue sections 
from the same tissue mass. The calibrate slide 101 
is prepared in the same manner as all other slides. 

75 but the tissue sections used are taken from a 
standard tissue mass having known amounts of 
estrogen and progesterone receptors. Preferably, 
all eight slides which are to be analyzed in the 
same session are lixed and stained in accordance 

20 with the above disclosed process as a batch, so 
that they all undergo substantially the same prep- 
aration. 

Upon completion of the fixing and staining of 
the slides 101 through 108. they are inserted into 

25 slide holder 62a which secures each slide at a 
predetermined position on the slide holder. In the 
present embodiment, calibration slide 101 is placed 
in the left most position and the slides 102 through 
108 are distributed in any order in the remaining 

30 seven slide positions. All slides are oriented so that 
their printed rectangle 87 is at the bottom as shown 
in FtG. 3. The slide holder is then inserted into the 
slide holder base 63a with the printed rectangles 
away from microscope 12. After placement of the 

35 slide holder, the operator signals, by means of 
keyboard 56, that the analysis is to begin. 

A flow diagram of the automated analysis rou- 
tine is shown in inter-related FIGS. 9 and 10. The 
routine begins at a step 136 with a request by the 

40 apparatus on instruction monitor 52 for patient la- 
belling information. When patient labelling is com- 
pleted by means of operator interaction with the 
keyboard 56. the automated portion of the analysis 
procedure begins and the operator is free to go 

45 about other tasks. 

Initially, microprocessor 36 by means of control 
signals sent over system bus 34 to the slide holder 
position controller 60. requests the movement of 
slide holder 62a to its maximum X and Y positions. 

so as represented by a point 120 in the upper left of 
slide 101 and a point 121 in the lower right of slide 
108. No actual marks appear on the slides to 
represent points 120 and 121. but these points are 
located by the maximum travel of the slide holder 

55 62a. The X and Y values of points 120 and 121 are 
read from position sensors 110 and 111 so that the 
position ranges of slide holder 62a are known by 
microprocessor 36. Since the slides 101 through 
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108 are held in predetermined positions, micropro- 
cessor 36 can easily compute the X address of the 
longitudinal center of each slide 101 through 108. 
and since the position of the rectangle 87 on each 
slide is also known the Y position of each rectangle 
87 can be computed with reasonable accuracy. 
These slide reference positions are calculated in 
step 140. The analysis routine next proceeds to 
step 142 in which the microprocessor 36 requests 
movement of the slide holder 62 to position the 
near center of rectangle 87 of slide 101 under the 
objective 64a. 

The automated analysis routine of the present 
embodiment is performed using a single 40X ob- 
jective which has a short depth of field on the order 
of a few microns. Although the slide holder and 
slides are relatively precise, they do not necessar- 
ily guarantee that the surface of the slides will 
always be at optimum focus when presented to the 
microscope objective 64a. Accordingly, as the ana- 
lysis for each new slide begins, the apparatus is 
first focused using a line 79 of the slide rectangle 
87. After the initial focus on line 79 is achieved, the 
apparatus is periodically refocused during analysis 
to provide accurate image fields. 

In a step 144, the slide 101 is moved so that 
the path of the objective travels along the imagi- 
nary center line 91 (FIG. 5) until line 79 is detected. 
Line 79 is of sufficient width and opacity that it is 
detectable even with a poorly focused objective. As 
the slide 101 moves, microprocessor 36 monitors 
the pixel array of image processor 90 which repre- 
sents the current image field presented to the 
objective. Microprocessor stops slide movement, 
when line 79 appears in the image field of image 
processor 90 and performs a focus routine on the 
inner edge of line 79. In the focus routine micropro- 
cessor 36 successively analyses the sharpness of 
the image field from image processor 40 and ad- 
justs the microscope stage to objective distance by 
means of focus and light controller 73. When ac- 
curate focus is detected by microprocessor 36. the 
flow proceeds to block 126 in which the slide is 
moved back to a point within rectangle 87. The 
rectangle 87 is to be free from any tissue section 
or other contaminants. Accordingly, the light level 
and color balance can be adjusted by means de- 
scribed above while the presented image field is 
from within rectangle 87. 

After light level and color balance are estab- 
lished the routine proceeds to a step 150 in which 
the slide is moved to view successive image fields 
along the center line 91 in search of a first image 
field representing reference tissue sample 89. The 
slide is moved so that the objective 64a traces a 
path represented by a search path line 94 shown in 
FIG. 5. Periodically slide motion is stopped, the 
objective 64a is focused by the microprocessor 36 



and the then current image field from image pro- 
cessor 90 is analyzed to determine if the control 
sample has been found. An image field of control 
sample 89 will be identified in a step 152 as an 

5 image field possessing analysis value. An image 
field is determined in step 152 to possess analysis 
value when the area of the nuclear material in the 
image field from image processor 90 exceeds a 
predetermined threshold stored in microprocessor 

10 36. Microprocessor 36 analyzes each image field 
pixel array from image processor 90 which arrays 
represent the nuclear image of the image fields. 
When the nuclear area of an image field exceeds 
the threshold stored in microprocessor 36. the im- 

is age field has analysis value and the flow proceeds 
to step 154 where attributes of the image field are 
measured and recorded. 

In step 154. the digitized image field from 
image processor 90 is analyzed to identify a nu- 

20 clear boundary level value for the nuclear material 
in the field. The nuclear boundary level which will 
be used in quantitation of the specimen 88 is 
stored in memory 38. Also, in step 154 micropro- 
cessor 36 measures the digitized image field from 

25 image processor 92 to identify an antitx)dy stain 
threshold. This threshold is necessary to determine 
the contribution of the non-specific staining of the 
control cells and the contribution of the counter- 
stain methyl green, to the total stain detected by 

30 the instrument. This antibody stain threshold per- 
mits the discrimination between antibody negative 
stain cells and antibody positive stain cells. After its 
determination, the antibody stain threshold level is 
stored in memory 38. Upon completion of the mea- 
ds, surement and recording of step 154. a step 156 is 
performed in which the digital image fields of both 
image processors 90 and 92 are stored in memory. 
The digital image fields may advantageously be 
stored in bulk data drive 45. 

40 The search path 93 (FIG. 5) begins within rec- 
tangle 87 and proceeds along center line 91 until a 
first image field having analysis value is detected 
at point 93. When microprocessor 36 detects the 
first control cell object image field, the search 

45 movement pattern, directed by microprocessor 36. 
changes the search movement pattern for image 
fields, as directed by microprocessor 36, follows a 
sweeping pattern back and forth across the control 
as represented in FIG, 5 by the line 94. While 

so traversing the reference specimen 89 successive 
image fields of the specimen are analyzed and for 
each image field the possessing analysis value, the 
nuclear boundary level and the antibody stain 
threshold are updated. Also, both digital images of 

55 each image field found to possess analysts value 
are stored in memory. Image fields continue to be 
analyzed from control specimen 89 until the total 
nuclear area of the image fields of analysis value 
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exceeds 5.000 square microns. When the 5.000 
square micron threshold is detected in step 158, 
slide holder movenr^ent is directed to return the 
objective 64 to center line 91 and to proceed 
downwardly (step 160) until a first specimen image 
field is detected at point 96 of FIG. 5. In a manner 
similar to step 152. the image field having analysis 
value is identified in step 164 of the flow diagram 
(FIG. 10) based on the nuclear area contained by 
the image field. 

When step 164 finds an image field having 
analysis value, the flow proceeds to step 166 
where attributes of the specimen image field are 
measured and recorded. The attributes measured 
and recorded in step 166 comprise the optical 
density of DAB areas 206 and 208. the visible area 
of DAB areas 206 and 208 and a comparison of the 
total nuclear area in the image field with the' area of 
DAB areas 206 and 208. After measuring and re- 
cording image field attributes in step 166. the 
digitized representations of the*image field are also 
stored in memory. In the present embodiment, the 
entirety of the specimen 88 is scanned by the 
apparatus as represented by the line 94 (FIG. 5). 
The completion of scanning for tissue sample 88 is 
detected in step 170. 

When, as is the case in the present example, 
the recorded data relates to calibration slide 101. 
as is detected in step 171. calibration values are 
calculated and stored tor later use in the analysis 
of measured attributes of other slides 102 through 
108. After the storage of the calibrate values in 
step 173, the flow proceeds to a step 172 to 
determine if all slides have been analyzed in step 
172. Since in the present example, they have not. 
the slide holder is moved to position the objective 
in the rectangle 87 of the next slide in the slide set, 
which is slide 102. In the present example, slides 
102 through 108 will be analyzed in sequence in 
the same manner as calibrate slide 101, except 
that the data measured in their specimen portions 
88 is not stored as calibrate data but as analysis 
results. 

When the last slide 108 has been completely 
analyzed, the flow diagram of FIGS. 9 and 10 is 
terminated and the accumulated analysis result 
data is available for reports as described in detail 
in the aforementioned United States Patent No. 
5.008,185 to Bacus. 

In the preceding embodiment, the cell samples 
were automatically located by the apparatus during 
an analysis operation. In order to speed analysis 
times, the apparatus 10 can be used to preselect 
points on the slide where analysis is to occur. The 
apparatus 10. includes the interactive optical means 
11a which comprises the microscope 12 having 
multiple turret objectives including a low power 
objective. The microscope 12 has mounted thereon 



a manually operable stage for carrying a micro- 
scope slide having a biological specimen thereon. 
The manually operable stage may be manipulated 
by the operator to bring various portions of the 

5 slide under the low power microscope objective. 
When the operator finds a region of interest, that 
region is located tDy the position sensors 12a which 
are disclosed in U.S. Patent No. 5.018.209 to 
Bacus. the content of which is incorporated herein 

10 by reference, may be marked by a press of a 
button 15 on the mouse, keyboard or the like to 
signal the processor means that the coordinates of 
that field are to be stored so that the field may later 
be examined at high power by the automatic op- 

15 ticat means lib. Thus, the slide may be quickly 
and rapidly scanned manually at low power select- 
ing the portions of interest and then loaded into the 
slide rack 62a with other slides for examination at 
high power by objective 64a using the high speed 

20 automatic processing as set forth previously. The 
low magnification scan of interactive system 1 1 a 
allows empty fields and fields which are not of 
interest to be avoided to reduce the amount of time 
consumed by the automatic optical means lib. 

25 The system 10 therefore provides the advantage of 
the combination of interactive, rapid, low power 
scanning and storage of coordinates with high pow- 
er automatic assay for full diagnostic treatment of 
the biological specimen. 

30 Further, after the high power assay has been 

completed, the images may be called up from the 
WORM drive 45 for additional editing or for dele- 
tion from assays, statistics and the like. Thus, the 
system allows pre-assay editing of the fields by the 

35 use of the low power microscope and post-assay 
editing of the fields with re-assaying by calling up 
the images from the WORM drive 45. 

In the preceding description, the tray of slides 
62a could be placed in the automatic optical ap- 

40 paratus 116 and the individual slides, e.g., 101. 
would be searched for calibrate and specimen 
samples, which samples would be appropriately 
analyzed. Additionally, the slides could be preview- 
ed under a low power microscope 12 and locations 

45 of analysis interest entered directly into the analy- 
sis apparatus 10 to identify the starting points of 
analysis functions. 

A third method and apparatus can also be 
employed to identify search locations on micro- 

50 scope slides. Tfie third method and apparatus, 
which is described in detail below, permits the 
selection of a plurality of search locations, called 
action points, on each of the plurality of slides in a 
slide tray, e.g., 62a. and permits the operator to 

ss specify an action e.g., scan or set light, at each 
selected action point. The action points and asso- 
ciated actions are easily selected interactively with 
the analysis apparatus 10 in preparation for an 
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analysis session. Advantageously, the action points 
and associated actions can be selected on other 
ancillary apparatus and a data file representing 
such selections can be prepared. When analysis is 
to occur, the slide tray is loaded into the automatic 
optical apparatus lib and a corresponding data file 
is entered into the analysis apparatus 10 by 
moveable storage means (diskette) or electronic 
communication. 

The selection of action points begins after at 
least one slide is mounted in a slide carrier (slide 
tray) 62a and involves visual inspection of the 
slides by an operator and entry of information by 
that operator into the computer 32 by means of 
keyboard 56 and mouse 13. When the slides have 
been loaded into the slide tray 62a. the tray is 
placed over a coordinate grid-forming template 251 
as shown in FIG. 14a and a tray setup function 250 
{FIG. 15) is initiated on the analysis apparatus 10. 
FIG. 14b shows a slide tray 62a held so that the 
coordinate grid of the template 251 is formed on 
the slides. The template 251 and slide tray 62a 
may be held in position by an appropriate jig (not 
shown). In the present embodiment, the slide tray 
62a is positioned over the gridded template which 
is viewed through the slides. Other methods of 
forming a grid reference on the slides could be 
used, such as projecting the grid on the slides by 
optic means or viewing the slides through a dear 
gridded template. 

The tray setup function includes operator re- 
viewing of its actual slides 101 through 108, a 
determination of the locations of points of interest 
(action points) on the slides from the coordinate 
pattern and the entry into apparatus 10 of informa- 
tion defining the points of interest. The tray setup 
action 250 begins with a visual image 270 (FIG. 16) 
of a slide tray and slides 273 being displayed on 
instruction monitor 52. Also displayed on monitor 
52 in the tray setup step (unction 250 is a tray 
setup menu 272. which identifies routines to be 
performed. The individual routines of menu 272 are 
selected by moving a cursor 271 by means of 
mouse 13 and selecting a routine by pressing the 
button 15 when the cursor is on a routine. The 
selection of menu items by means of a mouse and 
cursor is well known in the art. The routines avail- 
able in the tray setup function 250 are also shown 
in flow diagram form in FIG. 15. 

Initially, the label tray routine 252 is entered to 
establish the identity of the slide tray being set up 
for analysis. In the label tray routine 252, the in- 
struction monitor displays a window shown at 253 
of FIG. 15. requesting entry by the operator of a 
pre-assigned tray identity number and a pre-as- 
signed operator number. Upon exiting from the 
label tray routine by operator selection, the routine 
returns to the tray setup function 250 and estab- 



lishes a tray data file 280 (FIG. 17) for the identified 
tray. The data file 280 is stored in RAM 38 at a 
location identified by the tray identity number. The 
tray data file 280 includes a tray header 281 which 

5 stores both the tray identity number and the oper- 
ator identity number as entered by the operator. 

After the flow returns to the tray setup function 
250. the operator selects the label slide routine 
255. In the label slide routine 255, the operator 

to selects from the displayed slide tray one of the 
slide images 273 which corresponds to an actual 
slide in the slide tray being set up. The slide image 
273 is selected by the mouse 13 and cursor 271. 
Upon selecting a slide image, a prompt appears on 

75 monitor 52 for operator entry of the type of slide 
being set up. the patient number for the slide and 
the accession number. The prompt is represented 
at step 257 in FIG. 15. The slide types are shown 
at 274 in FIG. 16. When the data requested in step 

20 257 has been entered, the routine returns to the 
tray setup function 250 which stores the entered 
data in a slide header 282 of data structure 280 
(FIGt 17). The label slide routine 255 can be used 
to identify all 8 slides in the tray which resulting in 

25 the storage of 8 slide headers, each associated 
with one slide 101-108. In FIG. 16, the actual slide 
numbers from slides 101-108 have been written in 
parenthesis on slides 273 to show the correspon- 
dence between actual slides and slide images. 

30 After the slide images 273 have been labelled 
in routine 255, the operator selects the setup scan 
routine 258 from menu 272. The setup scan routine 
allows the operator to identify action points on the 
slides and to define an action to be performed at 

35 each identified action point. A flow diagram of the 
setup scan routine 258 is shown in FIG. 18. In the 
setup scan routine 258. the operator first selects a 
slide image 273 by cursor 271 movement. When a 
slide image 273 has been selected (step 300). the 

40 flow proceeds to a step 301 where the identity of 
the actual slide 103 corresponding to the selected 
slide image is read. Next, a step 302 is performed 
in which a slide grid 276 (FIG. 19), is displayed on 
monitor 52. Slide grid 276 is an enlarged image of 

45 the selected slide image and includes a reference 
grid pattern of four columns labelled A through D 
and nine rows, labelled 1 through 9. The rectangles 
represented in slide grid 276 correspond to the 
pattern of reference rectangles formed on the ac- 

so tual slides in the slide tray 62a. as shown in FIG 
14b. When the operator wants to specify a particu- 
lar point on the actual slide for analysis or other 
action, he or she selects the rectangle on slide grid 
276 which includes the particular point. Since the 

55 grid rectangles on the slide grid 276 correspond to 
the reference rectangles formed on the slides held 
in the slide holder (FIG. I4b). the identification of a 
rectangle of interest on slide grid 276 is a relatively 
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simple task. 

As represented in FIG. 14b. a portion of a cell 
sample is present in rectangle B-1 of slide 103. 
Accordingly, the operator moves cursor 271 to the 
rectangle B-l of slide grid 276 on monitor 52 and 
presses button 15 to select rectangle B-i. The 
rectangle position is read by the apparatus 10 at 
step 303 (FIG. 18) and the flow proceeds to a step 
304, where a large grid 277 and an action menu 

278 are displayed on nrionitor 52. Large grid 277 
represents the selected rectangle B-1 of slide grid 
276. and allows precise positioning of an action 
point within the selected rectangle. The large grid 
277 is sub-divided into, for example, 100 horizontal 
(X) by 100 vertical (Y) coordinate positions one of 
which can be selected by moving the tip of cursor 
271 thereto and pressing mouse button 15. A dot 

279 is shown in the lower left-hand quadrant of 
large grid 277 to represent a selected point in cell 
sample 263. The X, Y coordinate position of point 
279 in targe grid 277 is read by the apparatus in 
step 305. Since the position of rectangle B-l on the 
slide is predetermined and its position of point 279 
within rectangle B-l is now known, the absolute X, 
Y coordinate of the point on the slide is computed 
and stored in step 306. 

Each action point selected is associated with 
one of the actions listed in menu 278. Accordingly, 
the operator next moves the cursor to menu 278 to 
select one of the menu items. The selected action 
which is identified in step 307 is used in a manner 
described below to control apparatus 10 during the 
analysis of the slides on the slide tray. That is, 
after all slides have been appropriately "marked" 
with action points, each associated with an action, 
"the slide tray is mounted in automated assay ap- 
paratus lib as shown in FIGS. 1 and la and the 
apparatus 10 implements the specified actions at 
the specified action points. It is important to note 
that the actions do not take place when selected 
but are recorded in a data file, e.g., 280. for use 
after the slide tray 62a is placed in assay appara- 
tus lib. 

The following briefly describes the actions list- 
ed in menu 278 of FIG. 19. The show field action 
will cause the apparatus 1 1 b to align the specified 
action point with objective lens 64a and display a 
focused image field of the action point on image 
monitor 30. The joystick action will cause the ap- 
paratus to align the specified action point with the 
objective lens 64a and to release the control of the 
alignment between the objective 64a and the mi- 
croscope slide to the operator, through joystick 1 3a 
manipulation. The set and check light actions also 
cause the apparatus to properly align the specified 
action point with the objective lens 64a, at which 
point light levels can be set or checked respec- 
tively, as previously discussed. The scan action wilt 



cause the alignment of the microscope objective 
64a and the selected action point on a microscope 
slide and then begin to measure the attributes of a 
sample on the microscope slide. In the present 

5 embodiment, the scan action requires additional 
information to define the measurements to be 
made and how the scan is to be performed. The 
accumulation of such additional information is de- 
scribed later herein. 

10 A step 308 (FIG. 18) is performed to detect the 
selection of both an action point and its associated 
action. Subsequently, a check 309 is made to 
determine if the selected action is a scan action. 
Flow proceeds to step 311, when the selected 

15 action is not a scan action. In step 311. a dot 279 
is displayed on monitor 52 image 270 to indicate 
the position of the newly selected action point. 
Next, a step 312 is performed to store in data file 
280 the accumulated data, called a position record. 

20 identifying the coordinate address of the action 
point and its associated action. For non-scan ac- 
tions, the position record comprises data represent- 
ing the~location of the selected action point and the 
associated action. The position record is stored in 

?6 association with the slide header, e.g.. 282 of the 
slide identified in step 301. In data file 280 (FIG. 
17). the position records 283, 284 and 285 asso- 
ciated with slide 101 are shown occupying memory 
locations subsequent to the slide 101 header 282. 

30 It should be noted the other forms of association 
such as linking or address translation tables could 
be used to associate slide headers with their asso- 
ciated position records. Advantageously, multiple 
position records can be associated with the same 

35 slide. 

When the performance of step 309 detects that 
a scan action has been selected, the flow moves to 
step 313 in which a scan criteria menu 279 (FIG. 
20) is displayed on instruction monitor 52. The 

40 scan criteria menu 279 in conjunction with mouse 
and keyboard manipulation by the operator, defines 
the parameters of the scan to be performed at a 
previously selected action point. The scan type 
menu item 286 can be selected to set the scan- 

45 type to raster (as shown), horizontal or vertical. 
Raster scanning is a zig-zag pattern, as shown in 
FIG. 5. horizontal scanning consists ol a straight 
line across the slide and vertical scanning consists 
of a straight line along the slide length. The X and 

so Y step rate menu items 288 and 289 can be 
selected and set to define the spacing between 
evaluated image fields. Similarly, the definitions of 
blank fields and good fields (fields of analysis val- 
ue) can be established by selecting and setting 

55 menu items 290 and 291 . 

The sample type menu item 293, allows the 
operator to specify whether the sample being eval* 
uated is a specimen or a calibration sample. Speci* 
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tying the sample type identifies how the measured 
celt object attributes in the image fields are to be 
evaluated when the slide is later analyzed in the 
apparatus lib. The scan criteria menu 279 also 
includes a settable value 295 to identify the end of 
a scan line. As shown in menu 279, an end of line 
has been set to occur when three consecutive 
blank image fields are viewed, isienu 279 also 
includes a settable value 296 which terminates a 
raster scan at a preset number of zig-zags and a 
settable value 297 which terminates a scan when a 
set number of image fields have been analyzed. 

After the scan criteria have been set using the 
scan criteria menu 279, the operator exits from the 
menu and the apparatus reads, at step 314, the 
established scan criteria values. From step 314. 
flow proceeds to block 311 where the action points 
are displayed on the slide tray image 270. as 
previously described. Flow proceeds through step 
312. in which the position record is stored in data 
file 280. to the select slide step 300. It should be 
mentioned that for scan actions, the position record 
stored in data file 280 includes the scan defining 
criteria established in menu 279 in addition to the 
action point coordinates and specified action, e.g., 
scan. At the conclusion of the setup scan routine 
258. flow returns to the tray setup function 250 of 
FIG. 15. 

Four additional routines are available from the 
tray setup function 250. A load setup routine 259 
responds to the operator entry (step 260) of a slide 
tray identity, for which a data file was previously 
created, by reading (step 261) the identified slide 
tray data file. A save setup routine 262 commits a 
just-prepared data file 280 to a more secure mem- 
ory storage, for example, into disk storage. A clear 
setup routine 264 permits the clearing of the most 
recently created setup data file before it is commit- 
ted to the more secure storage. A delete setup 
routine 263 enables the deletion of an identified 
data file from the more secure memory area. 

After the completion of the slide tray setup 
function 250. a slide tray data file 280 as repre- 
sented in FIG. 17 exists in memory. The slide tray 
data file 280 is accessible by its tray identity num- 
ber and includes data representations of one or 
more action points, each in association with one of 
the described actions. 

FIG. 21 is a flow diagram of the steps per- 
formed by apparatus 10 in the analysis of biologi- 
cal specimens mounted on the slides of the iden- 
tified slide tray. The preceding actions establish a 
data file 280 identifying action points and actions 
for a slide tray identified by an identity number. 
When the slide tray is to be analyzed, it is loaded 
into automated assay processing system lib of 
apparatus 10 and the slide tray identity of the 
loaded slide tray is entered (step 320) at keyboard 



56. Initially, the slide tray ranges are found in step 
322 and slide positions calculated in step 324 in 
the manner described above with regard to the flow 
diagram of FIG. 9. Then, in response to the entered 

5 slide tray identity, the computer 32 retrieves (step 
326) the slide tray data file identified by the slide 
tray identity entered in step 320. Next, a step 327 
is performed in which the position records of all 
slides are read from the slide tray data file 280 and 

10 reviewed to identify all action points associated 
with joystick actions. The joystick actions are sepa- 
rated from the other actions because they involve 
continued operator activity. 

FIG. 22 is a flow diagram of the joystick com- 

15 pletion routine 328. Initially, the slide tray is moved 
(step 329) by the automated assay system lib to 
align the first joystick action point with the micro- 
scope objective 64a. After routine focusing, the 
image field at the action point is presented on 

20 display monitor 30 and the X-Y position control of 
the slide tray 62a is released to the joystick (step 
330). While the joystick 13a is in slide position 
control, the computer 32 responds to joystick sig- 
nals by controlling the assay processing system 

25 lib to move the slide to positions determined by 
operator interaction with the joystick 13a. When the 
operator finds, by observing the image on display 
monitor 30, an image field at which an action is 
desired, the mouse button 15 is pressed. The 

30 pressing of button 15 is detected in step 332. The 
pressing of the mouse button 15, while under 
joystick 13a control, establishes the image field 
present when the button is pressed as an action 
point. In response to the press of mouse button 15, 

35 an action selection menu is displayed on instruc- 
tion monitor 52 in step 334. This action selection 
menu is substantially identical to action selection 
menu 278 of FIG. 19. The mouse 13 is used to 
select in step 336 one of the listed actions for 

40 association with the action point then represented 
on display monitor 30. Upon selection of the action, 
a position record is created and stored (step 338) 
in the slide tray data file 280 (FIG. 17). After the 
new position record is stored in the data file, the 

45 list of joystick actions is checked in step 340 to 
determine if further joystick actions are present. 
When additional joystick actions are present, the 
flow returns to step 329 to perform again the 
above-described sequence. Alternatively » when no 

so joystick actions remain, the flow proceeds to action 
step 344 in which the performance of the actions 
now specified in slide tray data file 280 are per- 
formed. 

In action step 344, the slide headers e.g.. 282 
55 (FIG. 17) are reviewed in sequence from slide 101 
through slide 108 to identify slides to be analyzed. 
When a slide header for a slide to be analyzed is 
accessed, its associated position records are ac- 

13 
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cessed in sequence to control the operation of the 
analysis apparatus. In response to each position 
record in siide tray data file 280. the objective 64a 
Is substantially aligned with the action point defined 
in the position record, and the associated action 
stored in the position record is performed. When 
the action is a show field action 345. the assay 
processing system 1 1 b is controlled to present the 
image at the action point to display monitor 30 and 
to record a copy of the image. After such record- 
ing, flow returns to the action step 344 for the 
analysis of another position record. When a posi- 
tion record includes a set light or check light ac- 
tion, the set 346 or check light 347 function respec- 
tively is performed after movement of the appara- 
tus to the action point defined in the position 
record. The remaining action type is performed 
through action function 344 is the scan action 348. 
which is shown in greater detail in FIG. 23. 

The scan action begins at step 350, in which 
the slide tray 62a is controlled to move the image 
field at the selected action point to microscope 
objective 64a. As previously discussed, the field is 
focused after each move operation. The image field 
at the action point is then analyzed in step 352 to 
determine whether the image field has analysis 
value. The scan criteria of the present position 
record is consulted to determine the definition of a 
good field. For example, in the scan criteria repre- 
sented in the menu 279 (FIG. 20) a good field is 
one in which 70% of its area comprises cell ob- 
jects. When the image field does not have analysis 
value, the flow returns to block 350 where the slide 
tray is moved to place a new image field under 
objective 64a. The type of move operation per- 
formed is controlled by the previously established 
scan type. X step rate and Y step rate stored in the 
position record type for the present action point. 
For example, if the scan type is set to vertical 
scanning and the Y step rate to 2 apparatus lib 
would be controlled to move in the Y direction by 
two fields and stop to image a new field. After the 
new image field is focused, it is again analyzed in 
step 352 for a determination of its value for analy- 
sis. When the analysis value step 352 deterrriines 
that the current image field has analysis value, flow 
proceeds to step 354 where the attributes of the 
cell objects in the image field are measured and 
recorded. Next a pixel image of the current image 
field is stored in bulk data storage in step 356 and 
a step 358 is performed to determine if the end of 
the scan has been reached. The end of scan 
criteria are stored along with other scan criteria in 
the current position record. When the end of scan 
has not been reached, flow returns to step 350 
from which a new image field is found and ana- 
lyzed. When step 356 determines that the end of 
scan has been reached, a step 360 Is performed to 



determine the sample type of the cell objects being 
analyzed. If the sample type is a calibrate sample, 
the calibrate values are computed and stored in a 
step 362 and the flow returns to the action step 

5 344. Alternatively, when step 360 determines that a 
specimen sample has been scanned, flow returns 
directly to the action step 344. When in action step 
344. the computer 32 again reviews the slide tray 
data file to identify additional, unserviced position 

w records. Action step 344 will service these unser- 
viced position records until all have been serviced. 
At which time, a message is displayed on instruc- 
tion monitor 52 which indicates the completion of 
the present analysis operation. 

;6 In the preceding embodiment, the operator en- 
ters a sample type during the setting of scan 
criteria (FIG. 20). As specifically disclosed, the 
sample type is either specimen or calibrate. Other 
choices of sample type could also be available 

20 without departing from the invention. For example, 
the type of assay such as DNA mass assay or red 
blood cell hemoglobin assay, could be set as sam- 
ple types and the apparatus 10 could analyze the 
measured and recorded attributes accordingly. 

25 Also, each slide in a slide tray could include dif- 
ferent types of cell samples such as blood cells or 
tumor cells prepared in a manner specific to that 
type of celt sample. In such situations, the operator 
would enter the cell type during the label slide 

30 routine 255. The analysis apparatus would then 
read the sample type from the slide header, e.g., 
282 to cause the appropriate analysis routines to 
be executed. 

The disclosed embodiments present an auto- 

35 mated biological specimen analysis method and 
apparatus. Improvements are disclosed which per- 
mit an operator to preselect action points on one or 
more slides. The automated apparatus responds to 
the preselected action points by moving directly to 

40 them without the inefficiencies of a search routine. 
Advantageously, an action point pre-selection em- 
bodiment includes the ability to specify and record 
an action to be performed at each action point. 
Certain actions have been specifically disclosed, 

45 however, persons of ordinary skill in the art may 
include other actions without departing from the 
invention. Additionally, the disclosed embodiments 
use display monitors and menus for data entry. 
Other types, of data entry could be used without 

50 departing from the invention. 

While there has been illustrated and described 
a particular embodiment of the present invention, it 
will be appreciated that numerous changes and 
modifications will occur to those skilled in the art, 

55 and it is intended in the appended claims to cover 
all those changes and modifications which fall with- 
in the true spirit and scope of the present invention. 
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Claims 

1. A method for locating a small biological speci- 
men on a large specimen area of a transparent 
slide for later automated analysis of the speci- 
men, said method comprising the steps of: 

providing a slide tDefore an operator for 
viewing at low resolution so that the operator 
may rapidly exclude the empty area on the 
slide and proceed to locate visually the small 
specimen with the operator designating one 
section on the slide containing the specimen; 

viewing of the slide specimen area by op- 
erator at a first low resolution; 

viewing the one section containing the 
specimen at a predetermined high resolution 
and noting a marked portion in the section 
associated with the specimen and storing the 
marked portion as a coordinate address for 
later automated scanning at a substantially 
higher resolution than the predetermined reso- 
lution. 

2. A method in accordance with Claim 1 including 
the steps of: 

projecting a grid on the slide to divide the 
slide into a plurality of designated grid sec- 
tions, and selecting one of the designated grid 
sections having the specimen for viewing at 
the higher predetermined resolution. 

3. - A method in accordance with Claim 1 in which 

the noting of a marked position includes the 
steps of moving a mark on a monitor screen to 
the location to* be marked and then entering 
the address of the mark for a subsequent 
scanning operation. 

4. A method of analysis of biological specimens 
on each of a plurality of slides having large 
specimen areas with a specimen occupying 
only a small fraction of the specimen area, 
said method comprising the steps of: 

providing a plurality of specimen locating 
stations at which an operator locates the small 
specimen in the large specimen area; 

providing a specimen carrier at each sta- 
tion for the operator to handle a plurality of 
slides associates with the carrier: 

the operator at each station viewing a 
specimen slide at low resolution and the oper- 
ator storing the location of the specimen on a 
storage media; 

transferring the slides and their carrier to 
an automated analysis station; 

transferring the stored locations of the ad- 
dresses of the specimens to the analysis sta- 
tion; 



initiating scanning of the specimens at a 
substantially higher resolution than the tow res- 
olution at the location address previously es- 
tablished by the operator; and 

5 automatically positioning each of the slides 

beneath the microscope of a digital imaging 
apparatus for an automated analysis thereof 
using the previously applied addresses to 
avoid scanning large areas empty of the speci- 

10 men to be scanned. 

5. An interactive method of specifying one or 
more action points on each of a group of 
microscope slides mounted in a carrier com- 

75 prising: 

positioning a slide mounted in the carrier 
before an operator who visually inspects the 
slide to select the coordinate locations of one 
or more action points on the slide; 

20 entering into storage the coordinate ad- 

dress of at least one action point; and 

proceeding to the next slide of the group 
of slides in the carrier and having the operator 
enter the coordinate location for each action 

25 point into storage. 

6. A method in accordance with Claim 5 including 
displaying on a monitor an image of a micro- 
scope slide having a coordinate grid pattern of 

30 rectangles thereon; 

the entering step includes selecting one of 
the rectangles; 

forming an enlarged image on a monitor 
related to the location of the action point and 
35 the operator entering from the enlarged image 

a more finely defined coordinate address for a 
given action point into storage; 

forming a coordinate grid on at least one 
of the microscope slides and the operator us- 
40 ing a display cursor to locate the action point 

on the monitor grid related to the location of 
the action point on the slide; and 

selecting with the cursor one of the dis- 
played grid rectangles and then enlarging that 
45 grid rectangle for viewing on a monitor and 

using a spot locator to select a location spot 
within the enlarged grid rectangle on the moni- 
tor as the coordinate location of the action 
point. 

50 

7. A method of assaying with a microscope and 
an image analysis means of biological speci- 
mens on each of a group of microscope slides 
mounted in a carrier comprising: 

55 positioning a slide t>efore an operator who 

visually inspects the slide for one or more 
action points on the slide and views an asso- 
ciated coordinate system for the slide to select 
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a coordinate address on the slide for the action 
point: 

entering into storage the address and a 
selected function from one of several functions 
performable at the various action points: 5 

proceeding to the next slide of the group 
of slides in the carrier and having the operator 
view the slide and action point and an asso* 
date coordinate sysiem. the operator entering 
the coordinate address and selected function w 
for each action point into storage; and then 
proceeding to analyze the other slides in the 
carrier in this manner; 

positioning the carrier with the slides 
therein into an operative position with respect is 
to the microscope of the image analysis 
means; and 

automatically moving the carrier relative to 
the microscope to position each stored ad- 
dress on each slide at the microscope for 20 
performing the associated stored function ac- 
tion for that action point by the image analysis 
means. 

A method in accordance with Claim 7 includ- 25 
ing: 

displaying an enlarged image on a monitor 
related to the location of the action point and 
the operator entering from the enlarged image 
a more finely defined coordinate address for a 30 
given action point into storage; 

locating an action point for light calibration 
and locating an action point for analysis of 
specimen cells; 

forming a grid on the slide and for the 55 
operator viewing a grid displayed on the moni- 
tor from storage, the operator using a cursor to 
locate the address point on the grid on the 
monitor related to the location of the action 
point on the slide; and 40 

selecting with the cursor one of the grid 
areas and then enlarging that grid area for 
viewing on a monitor and using a spot locator 
to select a location spot within the enlarged 
grid area on the monitor as the coordinate 45 
address of the action point. 

An apparatus for interactively specifying one or 
more action points on each of a group of 
microscope slides, said apparatus comprising: so 

means for holding the group of slides in a 
side-byside relationship before an operator 
who visually inspects the slides for one or 
more action points on each slide; 

means for providing an associated coordi- 55 
nate system for the slide to allow an operator 
to select a coordinate address on the slide for 
each action point: and 



a storage means for storing addresses of 
selected action points on each slide. 

10. An apparatus in accordance with Claim 9 in- 
cluding: 

a monitor to display the coordinate system 
with respect to a slide at the operative position: 

means for displaying an enlarged image 
on the monitor related to the location of the 
action point: 

means for generating a spot on the en- 
larged image and thereby to produce a more 
finely defined coordinate address for a given 
action point; 

a grid for placing on the slide for viewing 
by the operator, a monitor having a grid dis- 
played from storage, a monitor cursor to locate 
the address point on the grid on the monitor 
related to the location of the action point on 
the slide: and 

means for producing from storage an en- 
larged view of a grid area on the monitor and a 
spot generator to select a location spot within 
the enlarged grid area on the monitor as the 
coordinate address. 

11. An apparatus for assaying biological speci- 
mens on each of a group of microscope slides, 
said apparatus comprising: 

means for holding a slide before an oper- 
ator who visually inspects the slide for one or 
more action points on the slide: 

means providing an associated coordinate 
system for the slide to allow an operator to 
select a coordinate address on the slide for the 
action point: 

a storage means for storing addresses and 
associated functions for each slide; 

means for entering into storage the ad- 
dress and a selected function from ^one of 
several functions performable at the various 
action points: 

said means for holding slide, holding the 
next slide of the group of slides in the carrier 
for viewing the next slide and action point 
thereon and an associated coordinate system 
therefor, said means for entering being op- 
erable by the operator to enter the coordinate 
address and selected function for each action 
point into storage: 

an image analysis means for receiving a 
carrier with the slides therein into an operative 
position; 

a microscope in the image analysis means 
for viewing a slide at the operative position; 
and 

means for automatically moving the carrier 
relative to the microscope to position each 
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stored address one each slide at the micro- 
scope operative position for perfornf)ing the 
associated stored function for that action point 
by the image analysis means. 

12. An apparatus in accordance with Claim 1 1 
including: 

a monitor to display the coordinate system 
with respect to a slide at the operative position; 

means for forming an enlarged image on 
the monitor related to the location of the action 
point: 

means for generating a spot on the en- 
larged image indication and thereby to pro- 
duce a more finely defined coordinate address 
for a given action point; 

said image analysis means includes 
means for performing a tight calibration func- 
tion at one addressed action point on a slide 
and means for analyzing specimen cells at 
another addressed action point on a slide; 

a grid for placing on the slide for viewing 
by the operator, a monitor having a grid dis- 
played from storage, and a cursor to locate the 
address point on the grid on the monitor re- 
lated to the location of the action point on the 
slide; and 

means for producing from storage an en- 
larged view of a grid area on the monitor and a 
spot generator to select a location spot within 
the enlarged grid area on the monitor as the 
coordinate address of an action point. 

13. An apparatus for automated assay of biological 
specimens positioned on microscope slides 
comprising: 

means for specifying at least one action 
point on a microscope slide; 

means for defining an action to be per- 
formed at each action point specified by the 
specifying means; 

means for storing associated representa- 
tions of the locations of each specified action 
point and the defined action to be performed at 
the specified action point: 

means for reading a specified action point 
representation and the defined action repre- 
sentation associated therewith from the storage 
means; and 

automatic analysis means comprising a 
microscope objective, for aligning said micro- 
scope objective with the action point repre- 
sented by the action point representation read 
from said storage means and for performing 
the action represented by the action repre- 
sentation associated therewith. 



14. An apparatus in accordance with Claim 13 
wherein said means for specifying comprises 
display means for displaying a visual image of 
portions of a microscope slide; 

5 means responsive to operator interaction 

for moving a cursor on said displayed visual 
image; 

means for selecting an action location 
identified by said cursor; 
10 ' said display means displays location refer- 

ence markings on the displayed visual image; 

means for forming on the microscope slide 
a visual pattern of location reference markings 
substantially identical to the pattern of refer- 
;5 ence markings displayed on said displayed 

visual image. 

15. An apparatus in accordance with Claim 13 
wherein one of said definable actions com- 

20 prises a scan action and said means for defin- 

ing actions comprises means for accumulating 
information identifying a type of analysis to be 
performed by said automatic analysis means, 
and said means for storing, stores the informa- 

25 tion identifying the type of analysis in associ- 

ation with a specified action point; and 

wherein said type of analysis includes an 
estrogen/progesterone assay or a ONA mass 
assay. 

30 

16. An apparatus for identifying action points for 
the performance of analysis functions on bio- 
logical specimens positioned on microscope 
slides, comprising; 

35 means for specifying at least one action 

point on a microscope slide; 

means for defining an action to be per- 
formed at each action point specified by the 
specifying means: 
40 means for storing in a data file associated 

representations of each specified action point 
^ and the defined action to be performed at the 
specified action point. 

45 17. An apparatus in accordance with Claim 16 
wherein said means for specifying comprises 
display means for displaying a visual image of 
portions of a microscope slide; 

means responsive to operator interaction 

50 for moving a cursor on said displayed visual 
image; 

means for selecting an action point iden- 
tified by said cursor; 

said display means displays location refer- 
55 ence markings on the displayed visual image 

or displays a pattern of grid markings defining 
a plurality of rectangles on said displayed vi- 
sual image, and said means for specifying 

17 



33 



EP 0 557 371 m2 



comprises means for specifying one of said 
rectangles: and 

means for forming on the microscope slide 
a visual pattern of location reference markings 
substantially identical to the pattern of refer-, s 
ence markings displayed on said displayed 
visual image. 

18. A microscope slide viewing apparatus compris- 
ing: w 

a slide tray for securing a plurality of 
slides in predetermined positions, said slides 
being secured such that an upper surface 
thereof is substantially coplanar; 

reference marked means having a surface ^5 
with a plurality of substantially identical refer- 
ence patterns marked thereon; and 

means for holding said slide tray in sub- 
stantially fixed position with respect to said 
reference marked means such that one of said 20 
reference patterns can be viewed on each of 
said plurality of slides. 

19. An apparatus in accordance with Claim 18 
wherein said slide tray holds said slides in a 25 
side-by-side relationship and said reference 
marked pattern means has a plurality of refer- 
ence patterns in side-by-side relationship cor- 
responding to the side-by-side relationship of 
said slides in said tray. 30 



35 



40 



45 



so 



18 



EP 0 557 871 A2 




19 



EP 0 557 871 A2 




20 



EP 0 557 871 A2 




91 



EP 0 557 871 A2 



-4 



64a 



62a 



63a 





CONDENSER 


STEPPER 
MOTOR 



Z4 

85a I 



-5 



VOLTAGE 
REGULATOR^ 

Trt 



86a 



+V 



FOCUS 
STEPPER 
MOTOR 

75a 



FOCUS 
AND LIGHT 
CONTROL 



74 



r-^ 91 




102 



23 



EP 0 557 871 A2 



FIG-6 



100 



METHYL 



75 



TRANSMISSION % 



50 



25 





\ GREEN 












^DIAMINOBENZIDINE 




■ , /, 


, GREEN 


RED 

\filter 



400 450 500 550 600 620 

WAVELENGTH OF LIGHT (NANOMETERS) 



FIG-7 




PEROXIDASE 

ANTI-PEROXIDASE (RAT) 



ANTI-RAT Ig 
(GOAT) 

MONOCLONAL 
ANTI-ER 
(rot IgG) 



PRIMARY 



FIG-8 



134 



ANTI-RAT Ig 
(GOAT) 

130 




PEROXIDASE 

ANTI-PEROXIDASE (RAT) 



NORMAL 
ANTIBODY 
(rat IgG) 



CONTROL 



700 



94 



EP 0 557 871 A2 



FIG-9 




FIND 
SLIDE HOLDER 
RANGES 



I 



CALCULATE 

SLIDE 
POSITIONS 



I 



MOVE 
INSIDE 
RECTANGLE 



I 



DETECT 
LINE AND 
FOCUS 



140 



142 



144 



SET 


LIGHT 


AND 


COLOR 



146 



MOVE 
IMAGE FIELD 



150 




MEASURE AND 
RECORD 



I 



STORE IMAGE 




FIELDS 










EP 0 557 871 A2 




END 



26 



EP 0 557 871 A2 



FIG- 11 '\ 




FIG-12 




FIG-13 



200— « 



204 
208- 





GREEN 


FILTER 




N 

J 




— ^ 


1 


























-J /- 









206 

^-202 



?7 



EP 0 557 871 A2 







O 










CM ^--^ 


o 






^ OD 






< 


















m 






< 






o 


















< 






Q 












OD 






< 






Q 






W 












< 












o 






CO 






< 






o 






CJ 






OD 








o 




o 






o 








T 




CD 


CD 




< 









.CM 



in 



.CM 



CM 



'cm: 



E 



D 



D 



00 o> 



Q 
0 



0 



<r*CMrO^ tou>r^ooo» 



28 



EP 0 557 871 A2 



^ CM K*) ^ tf> tf> 



CO o> 



O 

o 

CD 



Q 



00 



o 
o 

CD 



□ 



^ csj >o <o rs. 



o 
o 

CD 



CO o» 

— w ■■ 



0 



CM K) ^ in«or^ooo> 



O 
m 
< 

Q 
O 

ca 



^cMio-^ lo CO a> 



^ CM >o ^ in<or^floq> 



CM 



o 

o 



D 



^ CM >o in <o 00 o> 



o 
o 
m 



□ 



- o 



• o 



CM 



^cMto^ m u> CO a> 



o 

CM- 
CO 



o 
o 
m 



e 



^cMHi'^ io<or^aoa» 



EP 0 557 871 A2 



CO- 
CM 



< ID 
UJ I— 
—I UJ 
U 00 



CO- 



on 



CO- 
CM 



o 
in 

CM 



Z 




oo 
in- 

CM 



§5 



< 

to 



I 

o 



m 

CM 



3*^ 



CO 

in- 

CM 



ui o 

UJ ^ 
CO 



CM 



UJ O . 



P 2 
I- <o 



O 

^ CO 

UJ UJ Ui 
1 13 o 

I/O Q- •< 



LO 

T 



CM 

in- 

CM 



3^ 



CM 



o o 
>- 

UJ < UJ 
oc Ql 



EF 0 557 871 A2 



iO oo a» o CM 
lO m iO m to to to 

CN CM CM CM <N CS» CN4 




31 



EP 0 557 871 A2 



280 



TRAY ID 



OPERATOR ID 



SLIDE 101 HEADER 



SLIDE 101 POSITI ON RECORD 1 

SLIDE 101 POSITION RECORD 2 

SLIDE 101 POSITION RECORD 3 

SLIDE 102 HEADER 



SLIDE 102 POSITION RECORD 1 



32 



EP n 55? A71 A2 



•18 ^ 



258 



SETUP SCAN 




303 



304 



DISPLAY 
SLIDE GRID 



READ GRID 
POSITION 



305 



DISPUY LARGE 
RECTANGLE AND 
ACTION LIST 



306 



I 



READ DOT 
LOCATION 



I 



COMPUTE SUDE 
COORDINATES 



307 



IDENTIFY 
MENU ITEM 



308 



POINT 
AND ACTION 
SELECTED 
? 

[Y 



DISPLAY 
CRITERIA MENU 



3U I 



311 



312 



READ SCAN 
CRITERIA 



DISPLAY 
POINT 



I 



STORE POSITION 
RECORD 



aa 



EP 0 557 871 A2 




CM 



T 



< 

Q 



< 

o 
o 



o 

LU 

to 



>^ X ^ 

OO -J 
X O O X < X 

en CO -» CO o o uj 








CM 






m 


C£> 




CO 




o 




















o 




















m 


X 


















< 
























CM 


to 




m 


CD 




CO 





CM 



CM 



EP 0 557 871 A2 



CM 



< 



< 

o 
to 



lo 



o 



CO 

< 



o 

UJ 
CO 




Q- Q_ a. 

< X >- CD 
O 
CO 



CD 
I 

O 




<o 00 o> O 0) 
00 00 CD 0> ^ 
CM CM 04 CM 




35 



EP 0 557 871 A2 



FIG-21 

320 



322 



326 



327 



328 



344 



345 



i 



£NTER SLIDE 
TRAY IDENTITY 



FIND SLIDE 
TRAY RANGES 



324 I I 



CALCUUTE 
SUDE POSITIONS 



I 



RETRIEVE 
SLIDE TRAY 
DATA FILE 



I 



IDENTIFY 
JOYSTICK 
ACTIONS 



I 



RESOLVE 
JOYSTICK 
ACTIONS 
(FIG-22) 



I 



FIG-22 



PERFORM ACTIONS 



346 



348 



SET 






SCAN 


UGHT 






(FIG-23) 












SHOW 
FIELD 



CHECK 
UGHT 



347 



329 



330 



MOVE TO 
JOYSTICK ACTION 
POINT 



RELEASE CONTROL 
TO JOYSTICK 




336 



338 



OlSPUY JOYSTICK 
ACTION MENU 



I 



CHOOSE ACTION 



RECORD LOCATION 
AND ACTION 
IN DATA FILE 




TO FIG-21 
STEP 344 



36 



EP 0 557 871 A2 



FIG-23 



350 



352 



334 



356 



358 



360 



MOVE 
IMAGE 
FIELD 



MEASURE AND 
RECORD 



I 



STORE IMAGE 
FIELD 




"^>^END^\ 
C OF SCAN > 


N 






^^^ALIBRATE^\ 


N 








COMPUTE AND 
STORE 
CALIBRATE VALUES 









RETURN 
FIG-2t 
STEP 344 



37 



® 



Europe isches Patentamt 
European Patent Office 
Office europeen des brevets 




(2) Publication number: 



0 557 871 A3 



EUROPEAN PATENT APPLICATION 



© Application number: 93102433.5 
@ Date of filing: 17.02.93 



® Int. CIS: COIN 15/14, G02B 21/00 



(g) Priority: 18.02.92 US 837051 

@ Date of publication of application: 
01.09.93 Bulletin 93/35 

@ Designated Contracting States: 
BE CH DE ES FR GB IT LI NL 

® Date of deferred publication of the search report: 
01.12.93 Bulletin 93/48 



0 Applicant: CELL ANALYSIS SYSTEMS, INC. 
909 South Route 83 
Elmhurst. Illinois 60126-4944(IIS) 

@ inventor: Bacus, James W, 
826 South Lincoln 
Hinsdale, Illinois 60521 (US) 
Inventor: Bacus, James V. 
2233 Highland Avenue, 
Apartment 1502 
Lombard, Illinois 60148(US) 
Inventor: Wagner, Robert E. 
3539 North Fremont 
Apartment 2 

Chicago, Illinois 60657(US) 

© Representative: Selling, GUnther, DipL-lng. et 
al 

Patentanwalte 

von Kreisler. Setting, Werner 
Postfach 10 22 41 
D-50462 Kdin (DE) 



0 Method and apparatus for automated assay of biological specimens. 



CO 



00 

in 
in 



0 An apparatus and method for automated assay 
of biological specimens positioned on microscope 
slides. The apparatus comprises an interactive op- 
tical subsystem (11a) for viewing the biological 
specimen on the slide and for producing an inter- 
active video signal corresponding to the viewed im- 
age. An automated optical subsystem (lib) includes 
a single high power microscope objective for scan- 
ning a rack of slides, portions of which having been 
previously identified for assay in the interactive op- 



tical means (iia). The system also includes a pro- 
cessor for processing the interactive and automatic 
video signals for the two optical subsystems. The 
processor receives the automatic video signal and 
performs biological assay functions upon it. A meth- 
od and apparatus are also disclosed for marking 
points for later analysis on the microscope slides 
and for associating an analysis function with each 
marked point. 
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